The art and ‘science’ of opera: composing, staging & designing new forms of interactive theatrical performance

New technologies, such as Virtual Reality (VR), Robotics and Artificial Intelligence (AI) are steadily having an impact upon the world of opera. The evolving use of performance-based software such as Ableton Live and Max/MSP has created new and exciting compositional techniques that intertwine theatrical and musical performance. This poster presents some initial work on the development of an opera using such technologies that is being composed by Kallionpää and Chamberlain

Under construction – contemporary opera in the crossroads between new aesthetics, techniques, and technologies

Despite of its long history, opera as an art form is constantly evolving. Composers have never lost their fascination about it and keep exploring with innovative aesthetics, techniques, and modes of expression. New technologies, such as Virtual Reality (VR), Robotics and Artificial Intelligence (AI) are steadily having an impact upon the world of opera. The evolving use of performance-based software such as Ableton Live and Max/MSP has created new and exciting compositional techniques that intertwine theatrical and musical performance. This paper presents some initial work on the development of an opera using such technologies that is being composed by Kallionpää and Chamberlain. Furthermore, it presents two composition case studies by Kallionpää: “She” (2017) and puppet opera “Croak” (2018), as well as their documentation within the world's first 360° 3D VR recordings with full spatial audio in third-order Ambisonics and the application of an unmixing paradigm for focusing and isolating individual voices

Composing and realising a game-like performance for disklavier and electronics

“Climb!” is a musical composition that combines the ideas of a classical virtuoso piece and a computer game. We present a case study of the composition process and realization of “Climb!”, written for Disklavier and a digital interactive engine, which was co-developed together with the musical score. Specifically, the engine combines a system for recognising and responding to musical trigger phrases along with a dynamic digital score renderer. This tool chain allows for the composer’s original scoring to include notational elements such as trigger phrases to be automatically extracted to auto-configure the engine for live performance. We reflect holistically on the development process to date and highlight the emerging challenges and opportunities. For example, this includes the potential for further developing the workflow around the scoring process and the ways in which support for musical triggers has shaped the compositional approach

Interaction, instruments and performance: HCI and the design of future music technologies

Rationale There has been little chance for researchers, performers and designers in the UK to come together in order to explore the use and design of new and evolving technologies for performance. This workshop examines the interplay between people, musical instruments, performance and technology. Now, more than ever technology is enabling us to augment the body, develop new ways to play and perform, and augment existing instruments that can span the physical and digital realms. By bringing together performers, artists, designers and researchers we aim to develop new understandings how we might design new performance technologies. Some Topics - Methods and Approaches; What are the methods and approaches that we can employ to understanding interaction and interplay in performance and what impact does technology have on this? - Sonic Augmentation; can performance and sound change the experiential attributes of places, e.g. make them more accessible, more playful? -Physical/digital augmentation; how can one augment one’s self or existing musical instruments and artifacts physically and digitally? - Meaning and Mediation; can people narrate or make sense and movement as part of performance – how does the audience understand this? - Mobility and Immobility; performance and movement, what are the dynamics of performing at rest or whilst mobile, how can technology supported co-located and distributed performance and reception? - Locating Content and Spatialisation; how is performance located, how does sound and performance become part of the spatial fabric and what software tools can support this? - Personalization and Reflection; how can people use new performance technologies to narrate and reflect upon experiences – both as performer and spectator? These are some tentative implications and questions that we expect to address in the workshop. Goals The main goal of the workshop is to bring people together to discuss the issues mentioned previously and to explore this emergent space. As part of Audio Mostly we would like to build this community and develop a network that would engender ongoing participation, debate, scholarship and collaboration. The workshop would also like to encourage early career researchers and PhD students to attend in order to grow the community

Designing the audience journey through repeated experiences

We report on the design, premiere and public evaluation of a multifaceted audience interface for a complex non-linear musical performance called Climb! which is particularly suited to being experienced more than once. This interface is designed to enable audiences to understand and appreciate the work, and integrates a physical instrument and staging, projected visuals, personal devices and an online archive. A public premiere concert comprising two performances of Climb! revealed how the audience reoriented to the second performance through growing understanding and comparison to the first. Using trajectories as an analytical framework for the audience ‘journey’ made apparent: how the trajectories of a single performance are embedded within the larger trajectories of a concert and the creative work as a whole; the distinctive demands of understanding and interpretation; and the potential of the archive in enabling appreciation across repeated performances

Contesting control: journeys through surrender, self-awareness and looseness of control in embodied interaction

As Human-Computer Interaction (HCI) engages with technologies that sense and actuate the body, there is a need to reconsider the human bodily experience. We present three case studies that each involve different forms of bodily experience: a breath-controlled amusement ride, a brain-controlled film, and an interactive musical duet with a physically actuated piano. We introduce a conceptual framework to describe how control becomes contested between human and computer in such experiences, using the three dimensions of: surrender of control, self-awareness of control, and looseness of control. We reveal how our experiences took users on journeys through control that traversed the space of these dimensions. We propose that our framework is not only relevant to playful cultural experiences, such as those charted in our case studies, but can also inform the design of embodied interaction more widely by emphasising the human experience of control when engaging with autonomous and bodily-focused systems, from future robots and vehicles to today’s gaze, speech and gestural interfaces

Enterovirus-associated changes in blood transcriptomic profiles of children with genetic susceptibility to type 1 diabetes

Aims/hypothesis Enterovirus infections have been associated with the development of type 1 diabetes in multiple studies, but little is known about enterovirus-induced responses in children at risk for developing type 1 diabetes. Our aim was to use genome-wide transcriptomics data to characterise enterovirus-associated changes in whole-blood samples from children with genetic susceptibility to type 1 diabetes. Methods Longitudinal whole-blood samples (356 samples in total) collected from 28 pairs of children at increased risk for developing type 1 diabetes were screened for the presence of enterovirus RNA. Seven of these samples were detected as enterovirus-positive, each of them collected from a different child, and transcriptomics data from these children were analysed to understand the individual-level responses associated with enterovirus infections. Transcript clusters with peaking or dropping expression at the time of enterovirus positivity were selected as the enterovirus-associated signals. Results Strong signs of activation of an interferon response were detected in four children at enterovirus positivity, while transcriptomic changes in the other three children indicated activation of adaptive immune responses. Additionally, a large proportion of the enterovirus-associated changes were specific to individuals. An enterovirus-induced signature was built using 339 genes peaking at enterovirus positivity in four of the children, and 77 of these genes were also upregulated in human peripheral blood mononuclear cells infected in vitro with different enteroviruses. These genes separated the four enterovirus-positive samples clearly from the remaining 352 blood samples analysed. Conclusions/interpretation We have, for the first time, identified enterovirus-associated transcriptomic profiles in whole-blood samples from children with genetic susceptibility to type 1 diabetes. Our results provide a starting point for understanding the individual responses to enterovirus infections in blood and their potential connection to the development of type 1 diabetes.Peer reviewe

Enterovirus-associated changes in blood transcriptomic profiles of children with genetic susceptibility to type 1 diabetes

Aims/hypothesis Enterovirus infections have been associated with the development of type 1 diabetes in multiple studies, but little is known about enterovirus-induced responses in children at risk for developing type 1 diabetes. Our aim was to use genome-wide transcriptomics data to characterise enterovirus-associated changes in whole-blood samples from children with genetic susceptibility to type 1 diabetes. Methods Longitudinal whole-blood samples (356 samples in total) collected from 28 pairs of children at increased risk for developing type 1 diabetes were screened for the presence of enterovirus RNA. Seven of these samples were detected as enterovirus-positive, each of them collected from a different child, and transcriptomics data from these children were analysed to understand the individual-level responses associated with enterovirus infections. Transcript clusters with peaking or dropping expression at the time of enterovirus positivity were selected as the enterovirus-associated signals. Results Strong signs of activation of an interferon response were detected in four children at enterovirus positivity, while transcriptomic changes in the other three children indicated activation of adaptive immune responses. Additionally, a large proportion of the enterovirus-associated changes were specific to individuals. An enterovirus-induced signature was built using 339 genes peaking at enterovirus positivity in four of the children, and 77 of these genes were also upregulated in human peripheral blood mononuclear cells infected in vitro with different enteroviruses. These genes separated the four enterovirus-positive samples clearly from the remaining 352 blood samples analysed. Conclusions/interpretation We have, for the first time, identified enterovirus-associated transcriptomic profiles in whole-blood samples from children with genetic susceptibility to type 1 diabetes. Our results provide a starting point for understanding the individual responses to enterovirus infections in blood and their potential connection to the development of type 1 diabetes.Peer reviewe

Designing the audience journey through repeated experiences

We report on the design, premiere and public evaluation of a multifaceted audience interface for a complex non-linear musical performance called Climb! which is particularly suited to being experienced more than once. This interface is designed to enable audiences to understand and appreciate the work, and integrates a physical instrument and staging, projected visuals, personal devices and an online archive. A public premiere concert comprising two performances of Climb! revealed how the audience reoriented to the second performance through growing understanding and comparison to the first. Using trajectories as an analytical framework for the audience ‘journey’ made apparent: how the trajectories of a single performance are embedded within the larger trajectories of a concert and the creative work as a whole; the distinctive demands of understanding and interpretation; and the potential of the archive in enabling appreciation across repeated performances

The design of future music technologies: ‘sounding out’ AI, immersive experiences & brain controlled interfaces

This paper outlines some of the issues that we will be discussing in the workshop “The Design of Future Music Technologies: ‘Sounding Out’ AI, Immersive Experiences & Brain Controlled Interfaces.” Musical creation, performance and consumption is at a crossroads, how will future technologies be affected by exciting and innovative new developments in artificial intelligence, immersive technologies and developing mechanisms for interfacing with music, such as Brain Controlled systems. In many respects this document acts as a mini survey, made up of supporting material, a bibliography of works and offers a series of quotes from work that has mainly emerged from the FAST Project – see: www.semanticaudio.co.uk